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METHOD AND DEVICE FOR PROCESSING DIGITAL IMAGES 

The present invention concerns a method and a device for processing digital 

images. 

5 It is known to display a sequence of images coded by means of digital 

data according to various modes, a mode of display being defined by a direction 
and a speed of display dependent on the number of images displayed. For 
example, a sequence comprising five successive images Mi, M2, M 3> M 4 et M 5 
may be displayed according to a mode corresponding to a direction and a 
10 speed which are such that the images Mi, M 2 , M 3 , M 4 et M 5 are displayed 
successively. 

According to another mode, the direction of display is reversed with 
respect to the first mode while the speed of display of the sequence is 
accelerated in such a way that only the images M 5l M 3 et M 1 are displayed 
15 successively. 

According to a last example, the speed of display of the sequence is zero 
so that an image, for example Mi, is displayed continuously. 

Additionally, it is known to code digital images of a sequence of images 
using the coding of a first image of the sequence to perform the coding of a 
20 second image of this sequence. Subsequently in the document, such images 
are said to be dependent. 

Coded images of a sequence may be dependent, for example, when this 
sequence is coded according to the Motion Pictures Expert Group (MPEG) 
standard. 

2 5 It is recalled that, in accordance with this MPEG standard, the coding of 

an image associates the coded image with one of the following three 
categories : 

- A category of so-called intra (I) images, an intra image being 
decodable and displayable without requiring the decoding of another image. 
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- A category of so-called predictive (P) images, a predictive image 
requiring the decoding of the intra or predictive image transmitted prior to this 
predictive image so as to be displayed, and 

- A category of so-called bi-predictive (B) images, a bi-predictive 
5 image requiring the decoding of the images displayed prior and posterior to this 

bi-predictive image so as to be displayed. 

The present invention results from the finding that the displaying of 
images according to a specified mode, that is to say according to a specified 
10 direction and specified speed, requires considerable manipulations of images 
and hence considerable resources in the decoding devices. 

For example, considering the coding of images according to the MPEG 
standard, the displaying of a bi-predictive image requires the decoding of an 
15 image displayed previously and of an image displayed subsequently to this bi- 
predictive image, these three decoded images being displayed when the mode 
of display of the terminal corresponds to a speed such that the entire set of 
coded images are displayed. 

Conversely, if the speed of display is such that the bi-predictive image is 
20 displayed while its prior and posterior images are not displayed, for example for 
accelerated display, these latter must nevertheless be transmitted and decoded 
so as to allow the displaying of the bi-predictive image. 

It is apparent that as a function of the mode of display, first, non- 
displayed, images must be transmitted and decoded so as to allow the 
25 decoding of second images. 

Similarly, when the mode of display is such that the entire set of coded 
images is displayed, a predictive image is decoded by means of an image 
previously transmitted and previously decoded so as to be displayed. 

Conversely, if the speed of display is accelerated so that an image prior 
3 0 to a predictive image should not be displayed, this prior image is nevertheless 
transmitted and decoded so as to allow the decoding of the predictive image. 



WO 2004/013813 



PCT/EP2003/007476 



According to a last example, when the direction of display corresponds to 
the order of transmission of these images, a predictive image is decoded with 
the aid of an image previously decoded so as to be displayed. 

However, if the direction of display is reversed, the decoding of a 
5 predictive image requires the decoding of the image transmitted prior to this 
predictive image even though this prior image may possibly be displayed only 
after the displaying of the predictive image. 

Stated otherwise, the order in which the coded images are processed is 
unsuited to a reversed mode of display, thereby making it complex to process 
10 these images and increasing the display lags. The processing actually requires 
an analysis of the structure of the stream, a rearrangement of the images, 
multiple decodings entailing numerous resources in respect of the memory 
capacity required and of the power of the processors employed. 

15 These drawbacks are made more acute when the coded images are 

transmitted by a server to one or more remote display terminals. Specifically, in 
addition to the drawbacks relating to the complexity of the decoding and to the 
lags required for this decoding, transmission resources are used to transmit first 
coded images, necessary for performing the decoding of second coded images, 

20 which are not displayed. 

The present invention remedies at least one of these drawbacks. It 
relates to a method of processing digital images, these images being 
transmitted from a server to a decoder using various modes of display, images 
25 being coded in a dependent manner such that the coding of a first image is 
used to code a second image. 

According to the invention, the order of transmission of the images from 
the server to the decoder is determined as a function of the dependence of 
these images and the mode of display requested by the decoder. 

30 
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By virtue of such a method, the transmission of digital images can 
potentially be adapted to the mode of display in such a way as possibly to allow 
streamlining of the processing to be performed in the decoding device. 

In the case where the coded images are transmitted by a server to a 
terminal, the invention may permit a limitation of the use of the transmission 
network by transmitting only a limited number of images. 

According to a preferred embodiment, the mode of display used by the 
decoder is transmitted to the server so that the server performs the 
transmission of the images as a function of the mode of display used by the 
decoder. 

This may permit a reduction in the number of images transmitted 
between the server and the decoder or decoders for which these images are 
intended. The decoders may thus indicate to the server the mode in which they 
are operating and thus the server adapts the transmission of the images as a 
function of this mode. 

According to a preferred embodiment, when the server transmits coded 
images to the decoder, the server in parallel therewith transmits a command 
associated with each image comprising information allowing the decoder to 
process each image. 

The images are transmitted from the server to the decoder in an order 
which no longer necessarily corresponds to the order of coding. In the case of 
images coded according to the MPEG-2 standard, the stream transmitted from 
the server to the decoder no longer complies with coding according to the 
MPEG-2 standard since the images are transmitted in a different order from the 
coding. Thus, the command transmitted with the data allows the decoder to 
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decode the images of the stream even if this stream no longer complies with the 
norm in which the data were coded. 

According to a preferred embodiment, the information comprises' 
5 a field indicating whether the said image is to be displayed, 

a field indicating whether the said image is to be decoded, 
a field indicating in which memory buffer of the decoder the said image is 
to be recorded after decoding, 

a field indicating the index numbers of the memory buffers containing 
10 images serving for the decoding of the said image, 

a field indicating the direction of display of the images. 

This information tells the decoder the way in which it should process the 
image that it receives and where it should record this image. 

15 

According to a preferred embodiment, when the terminal receives coded 
images and a command associated with each image, the command is 
transmitted to a first memory buffer while the coded images are transmitted to 
other memory buffers. 

20 

According to a preferred embodiment, the commands stored in the first 
memory buffer are processed as a function of their order of storage in the said 
memory buffer. 

25 According to a preferred embodiment, use is made of modes of display 

defined by a direction of display, that is to say the order in which images are 
displayed, as well as by a speed of display such that a variable number of 
images is displayed for one and the same processed group of images. 
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The digital decoders in charge of the decoding of video sequences are 
often obliged to process special modes such as fast forward, delayed playback, 
rewind. 

According to a preferred embodiment, when the decoder changes display 
mode, the size of the first memory buffer is reduced. 

In this way, it may be possible to adapt the depth of the memory buffer 
and for example, to decrease the depth of the memory buffer when the speed of 
display decreases or to increase the depth of the memory buffer when the 
speed of display increases. 

This may permit a decrease in the latency within the decoder and thus 
may permit an improvement in the reaction time of the decoder. This may prove 
to be particularly beneficial during a change of display mode. 

According to a preferred embodiment, when the decoder changes display 
mode, the first memory buffer is emptied. 

This may also permit a decrease in the latency within the decoder when 
a change of display mode is performed. 

The invention also relates to a device for processing digital images, 
comprising means of transmitting these images from a server to a decoder 
using various modes of display, the images being coded in a dependent manner 
such that the coding of a first image is used to code a second image. 

According to the invention, the said device comprises means of 
determining the order of transmission of the images from the server to the 
decoder as a function of the dependence of these images and the mode of 
display requested by the decoder. 
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Finally, it should be pointed out that a method in accordance with the 
invention may readily be implemented in a known decoder as described later. 

5 The invention also relates to a computer program product comprising 

program code instructions for the execution of the steps of the method of 
processing digital images according to one of the embodiments of the invention, 
when the said program is executed on a computer. 

Other characteristics and advantages of the invention will become 
apparent in the description given hereinbelow, by way of non-limiting example, 
with reference to the following figures : 

Figure 1a represents the order of display of the data in forward 
mode and in backward mode, 

Figures 1b, 1c and 1d represent an example of the order of 
transmission of the images according to various modes of display, 

Figure 2 represents a server and two terminals in accordance with 
an embodiment of the invention. 

Figure 1a represents a sequence of images coded according to the 
MPEG-2 standard. 

The images are grouped into a sequence (GOP, Group of Pictures), 
comprising images coded according to the three modes described previously, I, 
25 BorP. 

A sequence comprises 12 images in the embodiment represented. In 
Figures 1a, 1b, 1c and 1d, the images denoted I, B and P belong to a GOP, the 
images denoted l\ B' and P' belong to another GOP and the images denoted I", 
3 o B" and P" belong to yet another GOP. 
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When the mode of display is forward, the images are displayed in an 
order corresponding to the order of coding of the images, the image BO being 
displayed first followed by the images B1 , 12, B3....P1 1 ... 

5 When the mode of display is backward, one commences by first 

displaying the image P11, B10, B99,...,B0.... In this latter case, the images are 
therefore not displayed in the order in which they were coded, this entailing 
considerable decoding constraints and considerable resources in a decoding 
device intended for decoding these images so as to transmit them to a display 
10 device. 

Represented in Figure 1b is a sequence of images transmitted from the 
server to the decoder according to an embodiment of the invention when the 
mode of display is forward. 

15 

In order to limit the necessary resources in the decoder, the images are 
transmitted in an order making it possible to decrease the processing in the 
decoder. 

20 Thus, the images are transmitted in the following order: 12, B0, B1, P5, 
B3, B4, P8, B6, B7, P11.B9.B10 

Represented in Figure 1c is a sequence of images transmitted from the 
server to the decoder according to an embodiment of the invention when the 
25 mode of display is forward accelerated. 

The type B images are not transmitted to the decoder. The I and P 
images are transmitted from the server to the decoder in the following order : 12, 
P5, P8, P11.... 

30 
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Represented in Figure 1d is a sequence of images transmitted from the 
server to the decoder according to an embodiment of the invention when the 
mode of display is backward. 

5 The images of one and the same GOP are transmitted in the direction : 

12, P5, P8, B10, B9, 12, P5, B7, B6, 12, B4, B3. 

It is therefore found that certain images (those for which a cross is 
indicated under the sequence) are transmitted several times so as to facilitate 
10 the processing in the decoder. 



Represented in Figure 2 are a server 202 and terminals 206 and 208 
intended for displaying images recorded in a recording medium (hard disk, CD, 
DVD) 204 of the server 202. 

15 

In accordance with the invention, the server 202 transmits images Mf 
coded in a dependent manner, that is to say that a first image Mj' is coded by 
means of a second coded image M k ', to the terminals 206 and 208. 

2 0 Additionally, these images Mj* are transmitted with a command Cj' 

identifying a mode of display associated with the images IvV, a mode of display 
being defined inter alia by a direction and a speed of display. 

To perform such a transmission of coded images Mi' and of command 
25 Q', information A 20 6 relating to the mode of display selected by the user of the 
terminal is transmitted to the server 204 by the terminal 206. 

As a function of the mode of display used by the terminal 206, various 
sequences of images as represented in Figures 1b, 1c and 1d, are transmitted 
30 to the terminal 206 to optimize the processing of these images. 
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The commands C'i et C"i transmitted by the server 202 to the devices 
206 and 208 comprise information relating to the decodings and to the displays 
to be performed by the devices 206 and 208 in such a way as to allow the latter 
to perform monitoring of the processing to be performed on the images Mr, M"i 
and on the commands C f \ CV' transmitted. 

The table 1 below illustrates the various information (the list is not exhaustive) 
of the commands C'i et C"i. 



Name 


Description 


RDCId 


Command identifier 


Display 


Indicates whether the image is to be displayed 


BuffNr 


Indicates the memory buffer in which the image is to be recorded 
after decoding. 


Decode 


Indicates whether the image is to be decoded 


P_BuffNr 


Indicates the index number of the memory buffer containing a P 
image which may be necessary to decode the current image 


LBuffNr 


Indicates the index number of the memory buffer containing an I 
image which may be necessary to decode the current image 


Forward 


Indicates the direction of display. This item of information is used 
by the decoder to display two imaaes in the interlpavprl mnHp 


Optional 


Size 


Size of the image (number of TS packets) 


Type 


Type of the image 


TempRef 


Time reference of the imaqe. 


commandeld 


Indicates a command index number making it possible to 
ascertain the mode of display to which the transmitted image 
corresponds. 



On the basis of this information, the server 202 models the operation of 
terminal 206, in particular in terms of available memory capacity, in such a 
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way as not to transmit a quantity of data greater than the processing capacities, 
and than the storage resources of the terminal 206. 
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When the signal comprising the data Mr and the command CT are 
received by the terminal 206, the signal is processed by a demultiplexer 210 
transmitting the command C,' to a register 212 and the coded images Mj' to a 
storage memory 214. The register 212 is preferably embodied as a memory of 
FIFO type (the acronym standing forfirst-in first-out). 

In this embodiment, this memory comprises two submemories 214a and 
214b used to store, respectively, the audio information and the video 
information received. 

The storage register 212 transmits the commands CV to a controller 216 
in an order corresponding to the order of storage in this register 212. 

When a command Q' is received by the controller 216, the latter 
commands by means of an element 218 the transmission of coded images from 
the storage memory 214 to a memory buffer 220 used for the decoding of the 
images. The module 218 is notified of the arrival of a new image in the memory 
buffer 214b and is commanded by the controller 216 to start the decoding of an 
image in the reconstruction memory buffers 220. 
2 o The decoded images are transmitted to the display screen 222 by way of 

a display controller 200. 

In this preferred embodiment, four memory buffers 220 are used to boost 
the rate of processing of the images, in particular when a modification of the 
mode of display occurs. 



15 
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When the mode of display is modified, information A 206 is transmitted to 
the server 202 which, modifying the coding of the images as a function of the 
new display mode, transmits new coded images M" associated with this new 
mode of display by means of a command C". 



30 
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In a preferred embodiment, the register 212 has an adaptive depth that 
makes it possible to reduce the latency when the user changes the mode of 
display. Thus the commands recorded in the register 212 are eliminated and it 
is possible to go swiftly to the next mode of display. 

In a variant, the register 212 is emptied when there is a change of display 
mode, this also making it possible to reduce the latency during a change of 
display mode. 

The field Commandld makes it possible to ascertain the mode of display 
to which each image corresponds. This identifier is transmitted initially from the 
decoder 206 to the server 202 with the mode of display. Then, each image 
corresponding to this mode of display is transmitted according to the sequences 
illustrated in Figure 1b, 1c or 1d with the identifier Commandld. Thus, during a 
change of display mode, the decoder, when it receives images corresponding to 
the previous mode of display, can empty the register 212. 

In a variant, the memory buffers 220 are specific to a category of coded 
image, intra, bi-predictive or predictive. 

When the images are coded according to the MPEG-2 standard, the CC 
field (the acronym standing for « continuity counter ») contained in the transport 
type packets is no longer valid since the images are transmitted in a different 
order from the coding. To avoid discontinuities in the stream, it is conceivable to 
insert padding packets so as to reach the expected value of the CC field. 
According to another variant, it is also possible to ignore the CC field in the 
decoder. 

When the images are coded according to the MPEG-2 standard, the 
commands C'i, C'"i can be inserted into private sections by using a TS packet 
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with a particular packet identifier (PID). In this case, the decoder decodes the 
PAT fields and the private tables. 

The commands C'i and C"i can also be inserted into a packet of TS type 
specially created for this purpose which contains only an identifier and no data. 



